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ABSTRACT 



This paper reports on ReMOTE (Research Methods Online 
Teaching Environment) , a Robert Gordon University (Scotland) project focusing 
on the development of a World Wide Web (WWW) site devoted to the teaching of 
research methods. The aim of ReMOTE is to provide an infrastructure that 
allows direct links to specialist sources in order to enable the use of 
existing materials as "plug ins" when developing research methods courses. It 
will also allow interactive computer-assisted learning packages to be 
downloaded and used in teaching or self-study. The approach is consistent 
with the broader goals of making use of the Internet in order to enhance the 
quality of learning, educational opportunities, profit, student numbers, and 
staff productivity. The following topics related to the development of the 
WWW site are described in detail: (1) requests for WWW links to add to the 

collection; (2) locating additional resources by searching on the WWW; (3) 
design, including user and institutional goals, the hierarchical structure of 
the site, and graphic design identity and functionality; and (4) content, 
including directory entries, materials hosted on the server, and courses. 
Relevant web sites are noted, and three figures illustrate the structure of 
the site, the ReMOTE opening screen, and categories in the directory of WWW 
sites. (DLS ) 
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1. Introduction T0 THE educational resources 

INFORMATION CENTER (ERIC).” 

In January 1997 the ReMOTE ( Research Methods Online Teaching Environment) project received funding for 
a 12 month period, its purpose to develop a World Wide Web (WWW) site devoted to the teaching of Research 
Methods, unifying access to a wide range of teaching materials. 



2. Background and Rationale 

Virtually all UK Higher Education Institutions teach Research Methods, either at the latter stages of 
undergraduate study or in postgraduate programmes, where Research Methods is seen as a key component in 
preparing students to undertake study leading to an MA or MSc qualification. Typically the topic is taught as a 
programme of lectures and seminars. 

The key features of Research Methods which make it an appropriate area to develop as an Internet resource are 
that: 

• it is generic and transferable. The end product will be applicable throughout RGU and all HEIs across a 
wide range of courses. 

• because of the wide range of potential topics which it is designed to support, students find that they may 
have to concentrate their efforts unevenly on different components within Research Methods: some may, for 
example, require more detail on quantitative analysis, whilst others do not need to study this topic in such 
depth because it will not be applicable to the type of dissertations or projects in which they are engaged. It 
is envisaged that the basic units will be extensible to allow students to explore in greater depth areas of the 
subject which are of interest to them whilst clearly making them aware of the knowledge and procedural 
requirements for achieving credits on the course, by exploring web links to more specialist sources. As 
such, the system will be particularly useful on postgraduate programmes of study, where students may 
require simply to upgrade some specific skills or techniques and can do so on an Open Learning basis. 

• the environment in which the project will be operating (the WWW) is one which is intrinsically important 
in conducting practical research and the mechanism of delivery can be viewed as an important part of the 
learning process itself. 

It is therefore apparent that such a widely applicable skill as Research Methods, and such a ubiquitous medium 
of delivery as the WWW are ideally suited for the development of a resource centre which will be of global 
utility. 

Although not explicitly stated within the project’s remit, it was recognised early on that the approach adopted 
was consistent with the broader goals of making use of the Internet in order to enhance: 
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the quality of learning 






• educational opportunities 

• profit 

• student numbers and 

• staff productivity 1 

Currently there are several WWW resources available for teaching Research Methods or facilitating the 
research process, the former typically concentrate on a subset of skills appropriate to a particular discipline or a 
single aspect of Research Methods, while the latter (e.g. SOSIG 2 ) tend to offer sites of use to researchers, and a 
few selected sites for teaching the subject. Pachnowski et al. 3 provide a useful discussion of some of the sites 
available which provide databases and examples of surveys of resources (e.g. the US Census Bureau 
http://www.census.gov/) and the Gallup Organization http://www.gallup.com/) but does not provide any 
detailed guidance on application of these resources in teaching. Useful resources are also provided by a range 
of directories - the Educator’s Internet Yellow Pages 4 being a particularly good starting point. More 
specifically related to Research Methods work by Cozby 5 provides an excellent guide to resources - each 
chapter of the book directing the student to a wide range of web resources and there are a number of web sites 
which give detailed tutorials on specific Research Methods concerns or techniques (e.g. BeLue’s Choosing a 
research design http://trochim.human.comell.edu/tutorial/belue/belue.htm and Bum’s pages on Securing 
internal validity http://trochim.human.comell.edu/tutorial/bums/int.html). 

The aim of ReMOTE is to provide an infrastructure which allows direct link to specialist sources which will 
enable the use of existing materials as “plug ins” when developing tailored Research Methods courses e.g. the 
use of links to the existing materials in the same way as references to the sources would normally be given in 
paper based documents. It will also allow interactive CAL packages, which are available freely via the 
WWW, to be conveniently downloaded and used in teaching or self-study. The researchers will then go on to 
test the effectiveness of the approach and study the manner in which the ReMOTE pages are used by staff and 
students within the University. 



3. Development 

The development of the WWW site required the identification of a wide range of Research Methods materials 
suitable for inclusion. One of the main aims for the methodology of the development for this project was 
perceived to be the need to provide rapid delivery of a working model which would act as a catalyst in 
stimulating ideas and feedback from academic staff who were not necessarily aware of what was achievable in 
this medium, but who nonetheless had significant teaching resources which could be used in developing the 
directory of learning materials (a process known as Rapid Prototyping, favoured by courseware designers 6 and 
suited to the HTML development environment). Thus at an early stage, the deliverables of two previously 
funded faculty projects which were identified as covering significant portions of the subject area in question, 
were deemed to be suitable for conversion into WWW format: 

• Self Study Learning Package for students studying Research Methods in the social sciences: produced by 
the Robert Gordon University School of Public Administration and Law, and consisting of ten learning 
papers, each covering a discrete aspect of the subject. 

• Communication Skills Guide, produced by William Robb for the Faculty of Management, consisting of a 
series of lessons in presenting written and spoken information. 

Both packages were in printed format, but also existed in electronic format and would not therefore require 
laborious verbatim retyping. 



1 Uys, Philip: Supporting cyber students over the web: The on-line campus 
Of Wellington Polytechnic , 18th ICDE World Conference "The New Learning 
Environment: A Global Perspective", June 1997 

2 SOSIG: the Social Sciences Information Gateway, URL: http://www.sosig.ac.uk 

3 Pachnowkski, LM et al. Immediate Data: The World Wide Web as a Resource for Teaching Research 
Methods. URL http://junior.apk.net/jurczyk/eera.html 

4 Place, R. et al. Educator’s Internet Yellow Pages. Prentice-Hall, Englewood Cliffs, 1996. 

5 Cozby, Paul C. Methods in Behavioural Research. 6th edition. 1997. 

6 Wilson, B. G., Jonassen, D. H., & Cole, P. (1993). Cognitive approaches to instructional design. In G. M. 
Piskurich (Ed.), The ASTD handbook of instructional technology (pp. 21.1-21.22). New York: McGraw-Hill. 
Also available at: http://www.cudenver.edu/-bwilson 



Further materials developed elsewhere were also to be considered for inclusion on the server where permission 
and copyright clearance could be obtained. It was already known that some materials developed elsewhere 
already existed in electronic format and might be suited to conversion or at least dissemination via the WWW; 
the amount of material, developed in isolation by lecturers at diverse institutions can only be speculated at. The 
overall effect of this approach was successful with some individuals responding enthusiastically to the 
possibility of making their own notes and lectures available on the WWW. 



2. 1 Requests for WWW Links 

A significant further part of the final deliverable is a collection of WWW links to already-existing materials 
elsewhere on the WWW. In order to identify materials for evaluation and possible inclusion in the site, input 
was solicited from a variety of email discussion lists and online forums which have an interest in web site 
development and teaching Research Methods. An announcement of the project’s intentions, coupled with a 
request for contributions, was sent out to: 

♦ Mailbase: UK Higher Education’s electronic discussions host. A number of lists were identified which 
might have an interest in the development of such a resource, including: 

• lis-link@mailbase.ac.uk 

• lis-infoskills@mailbase.ac.uk, 

• computer-assisted-leaming@mailbase.ac.uk 

♦ Other email lists and electronic forums, including: 

• nettrain@listserv.acsu.buffalo.edu 

A number of responses to this message were received, generally of an enthusiastic nature, although most 
revolved around the theme of “please let me know when it’s ready”. However, several worthwhile resources on 
the WWW were communicated, and some sites and individuals promised to donate materials, or to look into 
the practicalities of doing so. Similar messages were sent to individual sources which it was known had 
produced materials which might be suited to WWW conversion e.g. the PROCARE project at Southampton 
University. As in the case of the individuals who responded to the initial request offering their own materials 
for the site, enthusiastic initial responses have been followed by long periods of silence: in the case of 
PROCARE, copyright clearance deliberations at board level have thus far prevented any progress. 



3.2 Searching on the WWW 

In addition to asking interest groups to provide references, further materials - finally forming the bulk of the 
directory - were located using WWW searching techniques. Many materials were located via previously located 
WWW sites, which often include a list of links to other related sites (“further reading”). This has the 
advantage of the implicit recommendation by the link-maker in referring to the materials, but also restricts the 
user to what the individual author has himself already found. 

Searching was therefore extended to Internet directories 7 and search engines 8 e.g. Yahoo, The Argus 
Clearinghouse, AltaVista, Excite, Infoseek. This is a less focused method of identifying suitable resources than 
by individual recommendations, but has the potential to identify a much wider range of resources, and the 
majority of the directory’s contents have been located by this method. 



3.3 Design 



7 Directories are here defined as those WWW indexing services which are chosen and classified by human 
indexers, including Yahoo (http://www.yahoo.com) and the Argus Clearinghouse 
(http://www.clearinghouse.com) 

8 Search engines are defined as those WWW indexing services which identify and index resources by purely 
automatic means, retrieving matching documents on the basis of keyword matches rather than defined 
categorisations e.g. AltaVista (http://www.altavista.com). Excite (http://www.excite.com) and Infoseek 
(http://www.infoseek.com) 
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The structural and graphical design of the site required careful consideration in order to achieve a consistent 
and logical appearance. The factors which required to be taken into consideration included the goals of the 
potential users and of the university itself: 

♦ User Goals: 

• Navigability: use of logical structures to maximise the ease of use with which a user can 
locate the information they require, and to prevent them becoming “lost” in a complex 
structure of information 

• Clarity: the need for all of the possibilities to be presented in an intuitive and visible manner, 
making it clear at each stage - without the need for undue searching or experimentation on 
the user’s part - exactly what information can be reached from the current point 

• Attention to design: with all of the alternative sources of information on the WWW, it must 
be made obvious to the user that the resource has not been developed as a part-time “pet 
project” by an individual. A professionally designed look will: 

• be appealing on a visual level 

• engender confidence that this is a quality resource 

♦ Institutional Goals: 

• the need to establish ownership and a corporate identity; to remind users (albeit sub- or semi- 
consciously) that the Robert Gordon University is providing this resource 

• to provide a robust structure for the resource which will accommodate future changes and 
additions to the site in a logical manner without the need for excessive re-working of existing 
structures 

• to provide quality information in a manner which: 

• enhances the pedagogical aims of the university 

• enhances the profile of the university as an innovator in the use of novel techniques 
for information delivery 

On a practical level, these issues fall into two broad categories: 

• Structural design 

• Graphic design 



3. 3. 1 Structure of the Site 

The site is built on a hierarchical plan, where the most general resources contain links to the more specific 
ones. The home page serves to inform users of the purpose of the site and give access to the broad categories 
(not to the actual materials). The pages on the next level down unify access to i) materials on the server; ii) 
online courses and iii) the list of categories of materials from the WWW. In the case of (iii), the actual WWW 
resources are accessed after choosing the category required. In the case of (ii), the individual courses refer to 
the materials from (i) and (ii). Pages on the top two level of the hierarchy link clearly to each other, so that a 
user does not have to retreat back to the home page every time a different section is required. 
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List of categories 



Fig 1: Structure of the site 



The materials which have been installed on the server are structured in a manner reflecting as far as is possible 
that of the original materials. There have been some necessary alterations in terms of navigation e.g. the 
addition of hypertext contents pages. 



3.3.2 Graphic Design 

3.3.2. 1 Graphic Design: Identity 

It was decided that the three essential elements to convey as clearly as possible to users were: 

♦ The pedagogical purpose of the site: i.e. the teaching of Research Methods skills 

♦ The use of technology (the WWW) to enable distance or independent learning 

♦ Institutional ownership of the resource 

With these in mind, the site’s opening page (see fig. 2) was designed to convey these immediately and 
intuitively: 

♦ The banner graphic uses considerable corporate and research-appropriate imagery: 

• the underlying graphic consists of photographs taken from the 1998 Prospectus which illustrate 
elements of research: themes represented include use of telephones, interpersonal skills, 
technology (computer keyboard, CD-ROMs, floppy discs), libraries, study, books, files etc. 

• The RGU shield and corporate colours are prominent 

♦ The title ReMOTE itself is an acronym constructed from the words “Research Methods Online Teaching 
Environment”. The use of the term “remote” also reinforces the distance learning aspect. 

♦ The site content is overlaid on a background consisting of the undulating margin seen on numerous RGU 
publications, with the individual elements of the RGU shield (castle, boar’s head, cog, sphere and torch) 
continuing to the right hand side. The undulating margin serves a functional purpose, containing the 
navigation “buttons” which enable the user to access the site 



These graphic themes run throughout the contents of the site, and provide a consistent look and feel: it is easy 
for a user to tell which pages are a part of the ReMOTE site. 
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Research Methods Online Teaching Environment 

This page is a gateway to Research Methods teaching and learning materials 
on the Internet. It may be used by educators who will have access to specific 
materials or a whole structured course. The materials may then be used in a 
taught or self-study manner. If you have any good materials or links, or if you 
would just like more information, please contact me at the address below. 

This site consists of three sections: 

• A library of Research Methods links , open to the public 

• Materials held on this site : materials hosted on this server, developed at 
RGU and els where (requires password verification). 

• Courses developed for teaching research methods (will require 
password verification and a frames -cap able browser when released). 
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Fig 2: The ReMOTE opening screen 



3.32.2 Graphic Design: Functionality 

In addition to fostering a strong sense of purpose, technology and ownership (“PTO” identity), many of the 
graphics have a functional purpose. A decision was made that the non-use of graphics should, wherever 
possible, not inhibit use of the site: non-graphical browsers are still in use and browsing on standard browsers 
may be speeded by switching off graphics. Therefore functional graphics in the site are of a simple nature, 
generally only renditions of text with some embellishment e.g. 

♦ Navigation: the navigation buttons in the left hand margin, where the current page is highlighted in green 
with a tick beside it to make it absolutely clear to the user where they currently are in the structure. The 
functionality will be only marginally less clear if browsing in text only. 

♦ Pedagogy: in the actual learning materials, graphics do have an essential purpose. Where they are 
necessary, they have been kept to the minimum size for legibility on displays of varying resolution. 
However, the issues which can be discussed here are purely technical, since the use and design of these 
graphics are issues for the materials’ authors. 




3.4 



Content of ReMOTE: 



The contents will be dealt with in three parts: 

• The directory of links to other WWW materials 

• The materials actually hosted on the server (self study papers for the social sciences and the 
Communication Skills Guide) 

• Courses under development using the above materials 



3. 4 . 1 Directory entries 

Due to the extensive nature of the directory of links, it has been subdivided into categories for each aspect of 
Research Methods. Each category of the directory has a separate page, accessed from the main directory page: 



uvl 



Netscape - [Library of Research Methods Links] 



□ 1 X 



Fite Edit View Go Bookmarks* Options Directory Window Help 

S I C 1 1 



<>o 






ft 






*3 



C><3 



, Netsite: 



|http://jura2.eee.rgu. ac.uk/dsk5/research/libray.htm 


▼ ESI 


>1 -t — -rsr~- •” y-"~ . 7 ^ 


; 



School of Information and Media , The Robert Gordon University 



Kom-KI i .OKI * 'N 
UNI VI- RSJTY 

.M'- '• 'i I : n * 



ReMOTE 



Materials 

Courses 



Into 

Contact 




Links to research methods materials 

Last Updated: 06 February 1998 
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• Email resources 
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Fig 3: Categories in the directory of WWW links 

The format for directory entries is based on the style used in the NetLearn project 9 . The resource title is given 
as a hypertext link, followed by a brief abstract describing the resource, its coverage, target group etc. and 
finally some categorisation information. The information given should be enough to tell the user whether the 
resource is likely to contain the type of information which they might find useful (fig 4). 



9 NetLearn: a directory of resources for teaching and learning the internet. URL: 

http://www.rgu.ac.uk/sim/research/netlearn/callist.htm 






STEPS Project, University of Glasgow. Downloadable CAL modules for the teaching and learning of 
statistics, specifically applied to the following areas: Biology, Business, Geography, Psychology. 
Suitable for PCs, also some MAC material. The packages utilise graphic and interactive techniques. 
TYPE: materials, downloadable CAL software 

Fig 4: an example directory entry 



There are a number of drawbacks to the directory in its current form: 

• It is not searchable as a database would be. The name, URL, Description and categorisation information 
could all be used as fields in a database, but at present this is not possible. 

• Resources may fit into more than one category, resulting in duplication 

• Updating the directory must be done manually by editing the relevant HTML document: with the amount of 
information requiring updating, this is unwieldy and prone to errors. 

If the directory is to be maintained and promoted as a feature in the future, some investigation must be made 
into the possibility of converting the information into a database format accessible through a WWW front-end. 
This would not only make the user’s job searching for relevant materials easier, but also the maintainer’s job of 
adding, amending or deleting resources. Scope would then also exist for the addition of a “user comment” 
feature, where users could add their own annotations as to the usefulness of any individual resource. 



3.4.2 Materials hosted on the server 

The materials hosted on the server currently consist of two extensive packages which have been converted from 
paper-based versions: 

• Self Study Learning Papers for the Social Sciences (originally produced by the School of Public 
Administration and Law). This consists of ten learning papers, each focusing on a particular aspect of 
research. 

• Communication Skills Guide (originally produced by Mr. William Robb for the Faculty of Management) 

Both packages were deemed suitable for conversion to WWW format due, foremost, to their subject matter and 
the depth in which this was tackled. They complemented each other in that the former was a 10-step guide to 
doing research in the social sciences, while the latter presented an in depth tutorial on the presentation and 
communication of information. In addition to this, both already existed in electronic formats, having been 
created on word processing and desktop publishing programs, and would not therefore require laborious re- 
typing of the original text. Each package consisted of over 100 pages of mixed text and illustrations. 

First to be tackled was the Social Sciences package. The approach taken was to use a conversion utility 
(Microsoft Internet Assistant, or IA) which would take existing Word documents as input, converting them 
into HTML. The process of conversion with IA took only a few hours, but immediately presented problems: 
each individual Word document was converted into a single HTML document, which was not navigable and 
required to be broken into sections which would then have to be linked together. The HTML output by IA was 
also non-standard, and attempted a very literal interpretation of the look of the document when what was 
required was a representation of its structure and meaning. The parameters of paper and the WWW are not 
compatible, and the document has required considerable restructuring to make it manageable, and the 
inclusion of navigational facilities for locating and moving between documents - most notably tables of 
contents constructed of hypertext links to all of the available pages. 

A great number of further problems remained to be rectified due to the shortcomings of IA: many graphics had 
been discarded, others required re-sizing to make them legible on differing display resolutions and complex 
text formatting needed to be reinstated. An initial version was presentable after a week of work, but finding 
and correcting errors produced by IA required much longer, and the assistance of the original authors who 
were much more familiar with their material. The existence of a template into which content could be “poured” 
and links added automatically would have saved many days of effort: a single change in style often requires 
every page to be altered slightly, and there are well over 100 pages in the package. 
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The final product retains the original material’s content 100%, but represented by a different structure suitable 
for the WWW. The materials are broken down into smaller sections and direct access given to each through 
tables of contents (a device not used in the original materials). Although changes to the material were not 
within the scope of the project, in the future it is envisaged that they will gradually be altered to suit their new 
medium more closely, for example by the inclusion of links to outside WWW materials which might be 
referenced in the same way as texts would be in a conventional paper. 

The Communication Skills Guide required a different approach, as it had been created using Claris Works for 
the Apple Macintosh computer, which was not compatible with the PC software in use at SIM. The process 
was aided by the assistance of the Central Printing Unit who copied the text of each constituent file manually 
into a format which could be read by Word for Windows as used in SIM. The conversion was then undertaken 
by the manual addition of HTML tags. Navigational enhancements were made in a similar manner to the 
Social Sciences package, and some additional interactive and graphical features added which introduce a more 
personable style and stimulus throughout. 



3.4.3 Courses 

The final element of the site is the development of courses, tailored specifically towards teaching Research 
Methods in particular disciplines. Materials from the rest of the site may be built into a course, or a list of 
references, chosen by the lecturer concerned. These may be integrated with the lecturer’s own materials and 
used either as a replacement for or extension to normal modes of tuition: it may be prescribed as an activity or 
coursework, or it may simply be used for “extended references”. 

The use of the HTML frames facility is fundamental to this section. In using this, a left-hand column on the 
screen is used purely as an index for the actual materials. These materials are called up on the main portion of 
the screen when an item from the index is chosen. Therefore, if the majority of material to be used consists of 
existing WWW pages, there is a minimal requirement for writing HTML - this will consist of nothing more 
than a plain list of URLs. 

One of the project’s original stated aims was to develop such a resource for the Postgraduate Diploma in 
Information Analysis (PGIA). This is now under development: the standard course information has been put 
into HTML including the Course Unit Descriptor (CUD) and bibliography, plus references to the Social 
Sciences package, Communication Skills Guide and selected lecture notes/OHPs. Suitable areas for online 
study will be identified in co-operation with the lecturers responsible for teaching the Research Methods 
module and the resulting module is expected to be tested with students in the second semester of the 1997/8 
academic session. 



4. Next Steps 

The site will be announced to a wider audience, initially on a trial basis to elicit feedback and constructive 
criticism from a section of the academic community. The announcement will be made on a number of Mailbase 
electronic discussion lists including those previously mentioned, and to a number of selected individuals who 
are producing materials in the same area. Immediately prior to this, the site will undergo a process of revision 
in which consistency of style will be ensured, integrity of links checked and existing resources properly 
documented (there are some links which have not yet been fully investigated and abstracted). 

Testing with students will take place during 1998. Suitable online materials will be identified in conjunction 
with the relevant lecturers, and evaluation instruments applied to test the medium’s effectiveness. Evaluation 
checklists and questionnaires from the LTDI 10 have been obtained and will be adapted for use. 

The site will at all times continue to be extended, modified and enhanced in line with feedback from the user 
community and as new resources are discovered. 



4.1 Recommendations 



10 LTDI: Learning Technology Dissemination Initiative, established by SHEFC in August 1994 to support and 
encourage staff in the integration of technology with teaching. See URL: http://www.icbl.hw.ac.uk/Itdi 

10 



As noted above, the directory feature would be enhanced from the point of view of both of the user and 
maintainer by the integration of a database for storing the entries and a WWW front end which will make the 
directory searchable. This will enable greater flexibility, ease of use and functionality. The use of databases on 
the world wide web should be investigated further: it is envisaged that implementation of such a system would 
be inexpensive, the only major cost being the time which it would take to set up initially. 

Secondly, the requirement for consistency throughout the pages points to the requirement for a system of 
document management: currently, pages are constructed in a simple text editor with HTML encoding done by 
hand. This means that any change in the overall design of pages must be individually applied to each and every 
page affected, a repetitive and error prone process. Software which is capable of developing and using 
templates which can then be filled with appropriate content would eliminate a considerable amount of low- 
level repetitive work and make changes much more quickly and reliably. 

If large suites of web pages are in the future to be maintained with current information, both of the above 
issues must be considered: it will not be possible to compete with similar providers, nor to maintain extensive 
lists of up to date information without an automated system of document management 



5. Accessing the ReMOTE site 

At the time of writing the site is in a pre-release state: it has been announced to, and is in use by staff and 
students within RGU on an informal testing basis. Prior to announcing ReMOTE to the wider world, some 
additional resources are to be added, access restrictions placed on certain materials and the graphic aspects 
finalised. The site can be found at URL: http://jura2.eee.rgu.ac.uk/dsk5/research 
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